Difference in Differences Design

class: center, middle, inverse, title-slide

# Difference in Differences Design
## Tutorial 5
### Stanislav Avdeev

---

# Difference-in-differences

- The basic idea is to take fixed effects *and then compare the within variation across groups*
- We have a treated group that we observe *both before and after they're treated*
- And we have an untreated group
- The treated and control groups probably aren't identical! So... we *control for group* like with fixed effects
- Crucially, we need to have a group (or groups) that receives a treatment
- And, we need to observe them both *before* and *after* they get their treatment
- Observing each individual (or group) multiple times, kind of like we did with fixed effects
- The question DID tries to answer is "what was the effect of (some policy) on the people who were affected by it?"

---

# Difference-in-differences

- We can add a *control group* that did *not* get the treatment
- Then, any changes that are the result of *time* should show up for that control group, and we can get rid of them!
- The change for the treated group from before to after is because of both treatment and time. If we measure the time effect using our control, and subtract that out, we're left with just the effect of treatment!

---

# DID as two-way fixed effects

- We want an estimate that can take *within variation* for groups
- also adjusting for time effects
- and then compare that within variation across treated vs. control groups
- Sounds like a job for fixed effects

---

# DID as two-way fixed effects

- For standard DID where treatment goes into effect at a particular time, we can estimate DID with

$$ Y = \beta_i + \beta_t + \beta_1Treated + \varepsilon $$

- Where `$\beta_i$` is group fixed effects, `$\beta_t$` is time-period fixed effects, and `$Treated$` is a binary indicator equal to 1 if you are currently being treated (in the treated group and after treatment)
- `$Treated = TreatedGroup\times After$`
- Typically run with standard errors clusteed at the group level (why?)

---

# DID as two-way fixed effects

- Why this works is a bit easier to see if we limit it to a "2x2" DID (two groups, two time periods)

$$ Y = \beta_0 + \beta_1TreatedGroup + \beta_2Post + \beta_3TreatedGroup\times Post + \varepsilon $$

- `$\beta_1$` is prior-period group diff, `$\beta_2$` is shared time effect
- `$\beta_3$` is *how much bigger the `$TreatedGroup$` effect gets after treatment vs. before, i.e. how much the gap grows
- Difference-in-differences

---

# DID as two-way fixed effects

```r
set.seed(1500)
tb <- tibble(groups = sort(rep(1:10,600)), time = rep(sort(rep(1:6,100)),10)) %>%
  # Groups 6-10 are treated, time periods 4-6 are treated
  mutate(Treated = I(groups > 5) * I(time > 3)) %>%
  # True effect 5
  mutate(Y = groups + time + Treated*5 + rnorm(6000))
m <- feols(Y ~ Treated | groups + time, data = tb)
msummary(m, stars = TRUE, gof_omit = 'AIC|BIC|Lik|F|Pseudo|Adj')
```

<table style="NAborder-bottom: 0; width: auto !important; margin-left: auto; margin-right: auto;" class="table">
 <thead>
  <tr>
   <th style="text-align:left;">   </th>
   <th style="text-align:center;"> Model 1 </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Treated </td>
   <td style="text-align:center;"> 5.023*** </td>
  </tr>
  <tr>
   <td style="text-align:left;box-shadow: 0px 1px">  </td>
   <td style="text-align:center;box-shadow: 0px 1px"> (0.041) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Num.Obs. </td>
   <td style="text-align:center;"> 6000 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> R2 </td>
   <td style="text-align:center;"> 0.962 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> R2 Within </td>
   <td style="text-align:center;"> 0.609 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Std.Errors </td>
   <td style="text-align:center;"> Clustered (groups) </td>
  </tr>
</tbody>
<tfoot><tr><td style="padding: 0; " colspan="100%">
<sup></sup> + p &lt; 0.1, * p &lt; 0.05, ** p &lt; 0.01, *** p &lt; 0.001</td></tr></tfoot>
</table>

---

# Difference-in-differences

- We can use what we know about binary variables and interaction terms to get our DID

`$$Y_{it} = \beta_0 + \beta_1Post_t + \beta_2Treated_i + \beta_3Post_tTreated_i + \varepsilon_{it}$$`
where `$Post_t$` is a binary variable for being in the post-treatment period, and `$Treated_t$` is a binary variable for being in the treated group

---

# Difference-in-differences: simulation

```r
#Create our data
diddata <- tibble(year = sample(2002:2010,10000,replace=T),
                  group = sample(c('TreatedGroup','UntreatedGroup'),10000,replace=T)) %>%
  mutate(post = (year >= 2007)) %>%
  #Only let the treatment be applied to the treated group
  mutate(D = post*(group=='TreatedGroup')) %>%
  mutate(Y = 2*D + .5*year + rnorm(10000))
#Now, get before-after differences for both groups
means <- diddata %>% group_by(group,post) %>% summarize(Y=mean(Y))

#Before-after difference for untreated, has time effect only
bef.aft.untreated <-(means %>%  filter(group=='UntreatedGroup',post==1) %>% pull(Y)) - (means %>%  filter(group=='UntreatedGroup',post==0) %>% pull(Y))
#Before-after for treated, has time and treatment effect
bef.aft.treated <- (means %>%  filter(group=='TreatedGroup',post==1) %>% pull(Y)) -  (means %>%  filter(group=='TreatedGroup',post==0) %>% pull(Y))

#Difference-in-Difference! Take the Time + Treatment effect, and remove the Time effect
DID <- bef.aft.treated - bef.aft.untreated
DID
```

```
## [1] 2.002812
```

---

# Difference-in-differences

- How can we interpret this using what we know?

`$$Y_{it} = \beta_0 + \beta_1Post_t + \beta_2Treated_i + \beta_3Post_tTreated_i + \varepsilon_{it}$$`

- `$\beta_0$` is the prediction when `$Treated_i = 0$` and `$Post_t = 0$` `$\rightarrow$` the Untreated Before mean!
- `$\beta_1$` is the *difference between* Before and After for `$Treated_i = 0$` `$\rightarrow$` Untreated (After - Before)
- `$\beta_2$` is the *difference between* Treated and Untreated for `$Post_t = 0$` `$\rightarrow$` Before (Treated - Untreated)
- `$\beta_3$` is *how much bigger the Before-After difference* is for `$Treated_i = 1$` than for `$Treated_i = 0$` `$\rightarrow$` (Treated After - Before) - (Untreated After - Before) = DID!

---
# Difference-in-differences

- This is our way of controlling for time
- Of course, we're NOT accounting for the fact that our treatment and control groups may be different from each other
- Except that we are! We're comparing each group to itself over time (controlling for group, like fixed effects), and then comparing *those differences* between groups (controlling for time). The Difference in the Differences!
- Let's imagine there is an important difference between groups. We'll still get the same answer

---

# Difference-in-differences: simulation

```r
set.seed(1300)
#Create our data
diddata <- tibble(year = sample(2002:2010,10000,replace=T),
                  group = sample(c('TreatedGroup','UntreatedGroup'),10000,replace=T)) %>%
  mutate(post = (year >= 2007)) %>%
  #Only let the treatment be applied to the treated group
  mutate(D = post*(group=='TreatedGroup')) %>%
  mutate(Y = 2*D + .5*year + (group == 'TreatedGroup') +  rnorm(10000))
#Now, get before-after differences for both groups
means <- diddata %>% group_by(group,post) %>% summarize(Y=mean(Y))

#Difference-in-Difference! Take the Time + Treatment effect, and remove the Time effect
DID <- bef.aft.treated - bef.aft.untreated
DID
```

```
## [1] 2.0551
```

---

# Difference-in-differences: graphically

![](data:image/png;base64,#tutorial5_files/figure-html/unnamed-chunk-5-1.gif)

---